Recommender systems attempt to highlight items that a target user is likely to find interesting. For example, to recommend books to users of an online book store or to recommend users to other users. The items may be for example, goods, services, or users. Recommender systems may be used in a wide range of domains and a non-exhaustive list of examples is books, music, videos, images, web pages and news.
Many recommender systems use collaborative filtering which makes predictions about the tastes of a user (filtering) through collaboration among multiple users. Collaborative filtering systems predict whether a certain item is likely to interest a target user under the assumption that users whose past tastes are similar to the target user are likely to give a good prediction regarding the future tastes of the target user. For example, collaborative filtering systems may operate in two steps. First, they may seek users who share similar rating patterns with a target user. Second, they may use the ratings from those like-minded users found in the first step to generate a prediction for the target user.
It is difficult to build real-world collaborative filtering systems as these need to handle huge volumes of information in real time. For example, the universe of items is often enormous and there are many users. This has meant that many recommender systems have produced low quality recommendations because they are only able to use relatively small amounts of data. Recommender systems that operate by aggregating and correlating the opinions of many individuals show their true strength when using massive amounts of data. However, storing and processing such massive data sets is usually not tractable. Furthermore, in many cases the full data sets do not fit in memory, and thus have to be stored on secondary storage such as disks and tapes so that the processing is extremely slow.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known collaborative filtering systems.